Software Vault: The Gold Collection/Software Vault - The Gold Collection (American Databankers) (1993).ISO/cdr25/goserv.zip/GOSERVE.DOC

Text File | 1993-05-12 | 23KB | 509 lines

                                                               GoServe 1.00
GoServe -- A Gopher Server for OS/2 2.x
"""""""""""""""""""""""""""""""""""""""

Copyright (c) International Business Machines Corporation, 1993.
All rights reserved.


Introduction
""""""""""""
'Gopher' is a client-server protocol designed for text location and
retrieval.  It was designed at the University of Minnesota, and is
widely used; both clients and servers exist for most significant
operating systems.  The Gopher protocol is described in documents
distributed by the University of Minnesota Microcomputer and Workstation
Networks Center.

'GoServe' is a Gopher server for OS/2.  It is assumed that the reader is
familiar with Gopher concepts, and has access to a Gopher client.

GoServe allows processing of a variety of Gopher requests, using a
simple REXX filter to allow for customization.  Multiple servers can be
started (using different ports), and an audit trail of requests and
actions can be recorded.

While running, a Presentation Manager display of GoServe activity is
shown.  For control, a limit may be placed on the number of concurrent
requests that will be handled (hence limiting the load on the server
machine).  Similarly, timeouts may be set to close connections after a
chosen total or inactive time.  Certain GoServe operations can be
controlled remotely, if desired, using any Gopher client (or a utility
OS/2 program or CMD file).

Goserve is a 32-bit OS/2 application.  It requires TCP/IP for OS/2 to be
installed and operational (either on a real network or using the
loopback driver).

-----------

Mike Cowlishaw, IBM UK Laboratories


How GoServe is used
"""""""""""""""""""
When GoServe is started, it sets up a TCP/IP socket on port 70 (this is
the 'well-known' port for Gopher, but can be changed--see below), then
waits for requests from clients.  When a request from a Gopher client
arrives, it is recorded in the audit file (if desired) and then the
GoServe filter for the port is called.

The GoServe filter is a program written in REXX, which is given details
of the client and the selector string or strings.  The filter locates
the information (usually a file, containing a menu or a document) to be
sent to the client, and returns details to GoServe.  GoServe then sends
the data to the client and closes the TCP/IP connection.

Each incoming request is given a new thread for communication, filter
processing, and response.  GoServe handles all the details of threads
and TCP/IP communications, so the installer need only be concerned with
the filter (which can be very simple) and the data (menus and
documents).


Installation
""""""""""""
Only two files (plus data) are needed to run GoServe: the GOSERVE.EXE
program file, and the REXX filter.  The filter must have filename
GOFILTER, with an extension that is the TCP/IP port number to be used
(for example, 'GOFILTER.70' for the filter for the default Gopher port).

It is recommended that GOSERVE.EXE and the REXX filter be placed in one
directory (for example, 'D:\GoServe'), and data for the server be placed
in another (for example, 'D:\GoData70') or its subdirectories.  The REXX
filter should be in the working directory for GoServe, and any audit
file will be written to that directory.

The following files are included:

   goserve.doc  -- this document
   goserve.exe  -- the server program
   gofilter.70s -- sample filter (rename or copy as GOFILTER.70)
   gosmenu.cmd  -- menu maker utility (see below)

TCP/IP 1.2.1 (or later) must be installed; the "0.0024" CSD (or later)
for TCP/IP is required for most versions and betas of OS/2 2.x.  Over
time, other CSDs may become available--if TCP/IP problems are suspected,
always try the latest CSD.


Filter programs
"""""""""""""""
Filter programs are written in REXX.  Based on the arguments they are
passed, they decide on the menu or document to be sent to the client and
then return a result that defines that data to the server.

Three argument strings are passed to the filter:

1. The source of the request.  This has four words [more words could be
   added at the end in the future], separated by one or more blanks:

   a) the Internet address (primary host identifier) of the server's
      machine or cluster (in numeric form)

   b) the server's port number used for the connection

   c) the transaction number (a number, incremented for each new
      request, that starts at 0 when GoServe is started)

   d) the Internet address of the client (in numeric form).

   Example: "9.15.11.189 70 101 9.20.25.65"

2. The selector string.  If this is an empty string, it is the
   initial contact from a client, and the filter would normally respond
   with a main menu.  Otherwise, it will (or should) be the selector
   string from a Gopher menu line previously sent to the client,
   unchanged.

   You have complete freedom in setting selector strings in menus, and
   deciding how the filter should respond depending on the string
   received.  Often, the name of a file is used, and that file is sent
   to the client.  [Note: it is recommended that fully qualified
   filenames NOT be used for selector strings.]

3. The extended selector string.  Some Gopher clients can add additional
   information to a request, attached to the selector string after a
   Tab character.  GoServe discards the Tab character and presents the
   remainder as the third argument to the filter.  See the Gopher
   protocol documents for more information on how the extended selector
   string is used.

The filter can specify that GoServe send either a file or a single
string to the client, or request certain other GoServe operations. This
is done by returning a result string to GoServe.  This string takes one
of several forms (in which keywords and file specifications may be in
lower, upper, or mixed case):

1. FILE [ERASE] [BINARY] NAME filespec

   The file named by 'filespec' will be sent to the client.  'filespec'
   should normally be a fully qualified name (if it is not, GoServe will
   look for it in its working directory).  It may not include an
   'upwards reference' sequence ("..\"), as these could allow clients
   access to any file on the server machine.

   Either or both the optional keywords may be specified, in any order,
   and have the following effects:

   ERASE  -- the file is a temporary file, and should be erased after
             being sent.  [The transaction number can be used for
             generating a safe name for a temporary file.]

   BINARY -- the file is a binary file; after sending, no Gopher
             terminator line will be sent (and any trailing DOS
             end-of-file character will not be stripped off).  GoServe
             will close the TCP/IP connection as soon as the complete
             file has been sent.

2. STRING string

   The single string is returned to the client [along with added final
   carriage return, etc., as demanded by the Gopher protocol].  This can
   be used to return a simple message to the client.  Multiple lines can
   be sent, if necessary, by embedding a CR-LF sequence to separate
   lines.  A CR-LF is not needed at the end of the string.

3. CONTROL options

   A GoServe action (such as requesting statistics, resetting counters,
   or moving the audit file) is initiated.  This is used for controlling
   the GoServe server from a remote client or from another process on
   the same machine--for details, see the section "Remote control of
   GoServe".

   A document is returned, indicating success or failure of the request.
   Invalid options on CONTROL are not considered a failure of the filter.

4. Null string/no string

   If a null string, or no string, is returned by the filter, a one-line
   "no information available" document is sent to the client.

Any other response from the filter is not allowed; for safety, any
failure of the filter is considered catastrophic and will cause GoServe
to end.

Here is an example of a 'minimal' filter, GOFILTER.70:

  /* Sample GoServe filter program */
  parse arg source, selector, selector2                   /* Get arguments */
  parse var source server port transaction who .           /* Often useful */

  drive='d:'                                                 /* Data drive */
  dir=drive'\godata'port'\'   /* Data root directory (e.g. "d:\godata70\") */

  /* This very simple filter just builds the full path name of a file from */
  /* the selector and returns that.  Note we do not put the full path in   */
  /* menus, as this could allow clients to access all files on the server  */
  /* machine, which is rarely desirable.                                   */

  if selector='' then selector='mainmenu.'port           /* 'startup' case */
  return 'File name' dir''selector

Notes:

1. Remember that GoServe can run multiple copies of the filter at once,
   if more than one client connects at the same time.

2. The results of SAY instructions (or tracing) in the filter program may
   be seen using the PMprintf package.

3. You can execute OS/2 commands from the filter using the Address CMD
   instruction, for example:

     address cmd
     'start e foo.bar'
     'call d:\foobar\test.cmd argstring'

   Use caution when calling external programs: any slowness directly
   affects the response time to the client and its user.


Running GoServe
"""""""""""""""
GoServe may be started in any of the usual ways, such as from an OS/2
command line or from a Program Reference object.  By default, GoServe
uses the standard Gopher port (port 70).

Once GoServe has started, a PM window should appear.  If the window is
large enough, it will include at the top a bar display that shows any
client activity (the full width of the bar corresponds to the maximum
number of clients allowed).  A half-height grey bar indicates the peak
activity seen since GoServe was started.  This may be reset to zero by
selecting 'Reset peak indicator' from the 'Options' menu, and will be
shown in purple if any errors have been counted.

The second line of the window gives the total number of transactions
initiated since GoServe started, and the count of errors that have
occurred (if more than zero).  If room, it shows on the right the
instantaneous count of connected clients and the peak number recorded
since GoServe started (if more than one).  The transaction count may be
reset to zero by selecting 'Reset transaction count' from the 'Options'
menu).  Similarly, the error count and peak count may also be reset to
zero.

The third line shows the number of bytes, thousands of bytes, or
millions of bytes sent and received since GoServe started.  These counts
may be reset to zero by selecting 'Reset byte counts' from the 'Options'
menu.  The 'Options' menu also offers 'Reset all counts', which resets
all of the counters.

Some or all of these items will be omitted if the window is too small
for satisfactory display.  You can get more in a small GoServe window by
clicking on 'Use menu bar' on the 'Options' menu--this will remove the
menu bar from the display.  To use the menu after it has been removed,
click anywhere on the GoServe window with mouse button 2, in the usual
Workplace Shell manner (this works even if the menu bar is visible).

Two other menu options are supplied: 'Audit selection' lets you control
the auditing of transactions, and 'Limits selection' is used to control
the operating limits of the server.  These are described later.

The default port number [70] may be changed for special applications by
using the PORT parameter when starting GoServe, for example:

  start goserve port 71

Servers started with different ports are independent; they use different
filter programs, and maintain a different collection of settings, window
position information, audit file, etc.

Note that the port number is used as the required file extension for
some files--for port numbers greater than 999 you must therefore be sure
that the working directory for GoServe is on a disk that supports
extensions of more than three characters (for example, an HPFS disk).

GoServe also accepts two experimental keyword parameters.  'TRACE'
copies audit information to the PMprintf window (without GoServe
timestamps); this does not affect auditing to the audit file (see
below).  'DIAG' shows TRACE information together with additional details
of GoServe operation (also using PMprintf).  This diagnostic information
is intended for development use and may change over time.


The limits selection dialog
"""""""""""""""""""""""""""
The 'Limits selection' dialog lets you change the maximum number of
clients allowed concurrently [default 10], and the percentage at which
the bar chart will show a red warning indicator [default 75%].  It also
lets you change various timeout values.

The first two timeout settings control the time at which any uncompleted
response may be terminated automatically by GoServe's "watchdog" thread.

The first timeout setting is the 'inactive timeout', in seconds; after
at least the specified time without any data being sent or received the
connection will be closed [default 60 seconds].  The second is the
'total timeout', in seconds; after at least the specified time the
connection will be closed, even if there appears to be activity [default
600 seconds].  This value must not be less than the 'inactive' timeout.

For either timeout, a value of zero seconds may be specified to indicate
an indefinite time (that is, the timeout check is not made).  You should
only set both timeouts to zero if the filter is trusted, the data it
accesses is always available locally, and all clients used with the
server have a timeout mechanism.

Finally, a third timeout, the 'start timeout', specifies the time in
seconds for which GoServe will wait on startup for TCP/IP to become
operational, retrying appropriate TCP/IP calls as necessary [default 600
seconds].  This allows asynchronous starting of TCP/IP and GoServe.
Here, a value of 0 seconds means immediate timeout (that is, no retrys
are performed, and GoServe will not start if an error is found).

No retry is attempted if the requested port is in use (that is, it is
likely that another copy of GoServe is already running).


The audit mechanism
"""""""""""""""""""
GoServe includes a built-in audit mechanism that records events in an
audit file while Goserve is running.  This maintains a record of usage,
and also records any errors or failures of the server.

By default, all audit information (see details below) except selectors
is recorded.  Choose the 'Audit selection' menu item (under 'Options')
for a dialog that gives you detailed control over the content of the
audit file.  You should only select auditing of 'Selectors' if the
recording of selectors would not compromise confidentiality or privacy.

The audit file has a fixed filename (GOAUDIT), with an extension that is
the TCP/IP port number used (for example, 'GOAUDIT.70').  It is written
in the working directory for GoServe.  It may be read but not altered
while GoServe is running.

Each line in the audit file starts with three words, separated by one or
more blanks: the time (hhmmss), the audit type (one character), and the
transaction number (a number, incremented for each request accepted,
that begins at 0 when GoServe is started).  Additional information
follows, depending on the audit type:

  A (Accept)      - the numeric TCP/IP address of the client (n.n.n.n)
  C (Complete)    - transaction time (secs), and description of action
  D (Day)         - the current day (yyyymmdd), and description of audit
  E (Error)       - description of an error in processing (e.g., timeout)
  F (Failure)     - description of the failure (terminating error)
  I (Information) - description of information (statistics, etc.)
  P (Port)        - server identity and port (n.n.n.n nn)
  S (Selector)    - selector received from client

The format of any "description" in the above is not strictly defined.
Additional audit types may be added later.

For example:

  151717 D 0 19930512 ----- GoServe 1.00 -----
  151717 P 0 9.20.5.59 70
  151730 A 1 9.20.5.59
  151730 S 1
  151731 C 1 0.55 Sent file "d:\godata70\mainmenu.70"
  151733 A 2 9.20.5.59
  151733 S 2 about.doc
  151734 C 2 0.2 Sent file "d:\godata70\about.doc"
  151931 I 2 Bytes sent 99 received 11
  151931 I 2 Close

The transaction time recorded on 'C' lines is rounded to the nearest
50ms, and measures the total elapsed time from receiving the request to
complete delivery of the data.  This depends on the speed and activity
of the server, client, and the network between the two.  It is likely to
be rather variable.

To look at an 'active' log file, use a program or command that allows
shared reading of files.  TYPE is one such command; its output may be
redirected to a file to take a snapshot copy of the log.  For example:

  type goaudit.70 > snapshot.log

In addition, the 'Options' pull down menu has an item 'Move audit to
archive'.  This closes the audit file, appends it to the GoServe archive
file in the current working directory, then erases and re-opens the
audit file.  The GoServe filter can also initiate this action, by
returning "control moveaudit"--see elsewhere for details.

The GoServe archive file has a fixed filename (GOARCH), with an
extension that is the TCP/IP port number used (for example,
'GOARCH.70').  It is only touched by GoServe during a "Move audit to
archive", so is readily copied, edited, or renamed.


GOSMENU.CMD
"""""""""""
The GoServe package includes a simple utility that makes it easier to
create 'canonical' Gopher menu files.  See the start of GOSMENU.CMD for
details.  A 'menu template' file, perhaps called MAINMENU.MEN, might
look like this:

  Sample main menu
  ; rexx.central.edu 70
  0 About this Gopher server; about.doc
  1 IBM Almaden Gopher server;; almaden.gopher.ibm.com
  1 Interesting stuff; interest.70

The first line is a description; the second gives the field separator
character (which will become Tab in the generated menu), together with
the default server address and port.  Subsequent lines are menu lines,
as required by the Gopher protocol.  Lines whose first character is a
blank are ignored (that is, they are not included in the menu file).

Running GOSMENU against this file (with a different output filename)
like this:

  gosmenu mainmenu.men mainmenu.70

will create a 'canonical' menu file, with real tab characters instead of
semicolons, address and port added where required, and extra blanks
removed.  The Gopher protocol suggests that the first (description)
field be no more than 70 characters, and requires that the entire menu
line (including address and port) fit within 255 characters.

Notes:

1. The default output filename for GOSMENU.CMD is constructed by replacing
   the extension of the input filename by '70', so the sample command could
   have been simply:

     gosmenu mainmenu.men

   Or, of course, the file MAINMENU.MEN could have been dropped on a
   Program Reference object that starts GOSMENU.CMD

2. It is recommended that 'active' menu files be identified by an extension
   which is the Gopher port for which they are to be used.


Remote control of GoServe
"""""""""""""""""""""""""
GoServe filters (see "Filter programs") can request that GoServe perform
certain actions, instead of just returning files or other data.  These
requests may be used to control GoServe remotely (from another machine
or from another process on the same machine).  All requests return a
document of one or more lines, either showing successful completion of
the request or indicating an error.

Specifically, if the filter response string begins with the verb
"CONTROL", then the remainder (after one or more blanks) may take one
of several forms:

1. SAY text

   This is a simple "loopback" test: the specified text is returned to
   the sender as a document.

2. STATISTICS

   Current statistics and settings are returned as a document.  These
   include:

   * Transaction, error, byte, and client counts

   * Settings (not including for audit selections)

   * The local time of certain events (if an event has not occurred, it
     is shown as the GoServe start time).

   [The exact format of these statistics could change in the future.]

3. RESET [BYTES] [PEAK] [TRANSACTIONS] [ERRORS] [ALL]

   The specified GoServe counters and statistics are reset (after
   auditing, if appropriate).  Any or all of the keywords may be given,
   in any order.  Each works in the same way as the corresponding menu
   pull-down.

4. MOVEAUDIT

   The audit file is copied to the archive file (just as though the
   'Move audit to archive' menu item had been selected).

The filter mechanism provides flexibility in how these requests are
accepted and passed to GoServe.  For example, to accept control requests
(here identified by a selector that starts with '!') only from certain
addresses, the last two lines in the sample filter in "Filter programs"
could be replaced with:

  if left(selector,1)<>'!' then do        /* this is not a control request */
    if selector='' then selector='mainmenu.'port         /* 'startup' case */
    return 'File name' dir''selector               /* normal selector case */
    end
  /* this is a control request */
  owners=server 19.112.45.7                /* list of authorized addresses */
  if wordpos(who, owners)=0 then return    /* ignore unauthorized requests */
  return 'control' substr(selector,2)    /* remainder of string is request */

A control menu for issuing control requests might include lines like
these (prior to processing with GOSMENU.CMD):

  0 Show statistics;       !statistics
  0 Move audit to archive; !moveaudit
  0 Test loopback;         !say Hello world!
  0 Reset all;             !reset all
  0 Reset peak count;      !reset peak
  0 Reset byte counts;     !reset bytes
  0 Reset error count;     !reset errors
  0 Reset transactions;    !reset transactions

The filter could be further modified so that only clients with
authorized addresses would see a menu with these options.